Object detection and activity recognition in video surveillance using neural networks

Purpose: This paper aims to implement and extend the You Only Live Once (YOLO) algorithm for detection of objects and activities. The advantage of YOLO is that it only runs a neural network once to detect the objects in an image, which is why it is powerful and fast. Cameras are found at many different crossroads and locations, but video processing of the feed through an object detection algorithm allows determining and tracking what is captured. Video Surveillance has many applications such as Car Tracking and tracking of people related to crime prevention. This paper provides exhaustive comparison between the existing methods and proposed method. Proposed method is found to have highest object detection accuracy. Design/methodology/approach: The goal of this research is to develop a deep learning framework to automate the task of analyzing video footage through object detection in images. This framework processes video feed or image frames from CCTV, webcam or a DroidCam, which allows the camera in a mobile phone to be used as a webcam for a laptop. The object detection algorithm, with its model trained on a large data set of images, is able to load in each image given as an input, process the image and determine the categories of the matching objects that it finds. As a proof of concept, this research demonstrates the algorithm on images of several different objects. This research implements and extends the YOLO algorithm for detection of objects and activities. The advantage of YOLO is that it only runs a neural network once to detect the objects in an image, which is why it is powerful and fast. Cameras are found at many different crossroads and locations, but video processing of the feed through an object detection algorithm allows determining and tracking what is captured. For video surveillance of traffic cameras, this has many applications, such as car tracking and person tracking for crime prevention. In this research, the implemented algorithm with the proposed methodology is compared against several different prior existing methods in literature. The proposed method was found to have the highest object detection accuracy for object detection and activity recognition, better than other existing methods. Findings: The results indicate that the proposed deep learning–based model can be implemented in real-time for object detection and activity recognition. The added features of car crash detection, fall detection and social distancing detection can be used to implement a real-time video surveillance system that can help save lives and protect people. Such a real-time video surveillance system could be installed at street and traffic cameras and in CCTV systems. When this system would detect a car crash or a fatal human or pedestrian fall with injury, it can be programmed to send automatic messages to the nearest local police, emergency and fire stations. When this system would detect a social distancing violation, it can be programmed to inform the local authorities or sound an alarm with a warning message to alert the public to maintain their distance and avoid spreading their aerosol particles that may cause the spread of viruses, including the COVID-19 virus. Originality/value: This paper proposes an improved and augmented version of the YOLOv3 model that has been extended to perform activity recognition, such as car crash detection, human fall detection and social distancing detection. The proposed model is based on a deep learning convolutional neural network model used to detect objects in images. The model is trained using the widely used and publicly available Common Objects in Context data set. The proposed model, being an extension of YOLO, can be implemented for real-time object and activity recognition. The proposed model had higher accuracies for both large-scale and all-scale object detection. This proposed model also exceeded all the other previous methods that were compared in extending and augmenting the object detection to activity recognition. The proposed model resulted in the highest accuracy for car crash detection, fall detection and social distancing detection. © 2023, Emerald Publishing Limited.

Prototyping a VR Sandbox for Scene Customization without 3D Authoring Skills

Virtual Reality (VR) and simulation continue po-sitioning as suitable tools for fine-tuning processes otherwise impossible in real life. Such is the case of Aether, a mobile service robot for elderly care developed during the COVID-19 pandemic. Aether's development was negatively impacted due to restrictions placed on accessing long-term care facilities that impeded testing object tracking, elderly tracking, fall detection, and human-robot interactions. Our efforts to maximize Aether's development led us to create a digital twin where the core functionality is replicated to train the machine learning modules to optimize the robot's responses before real-world deployment. However, the digital twin creation requires significant authoring to ensure the virtual environment matches the real one by employing 3D technical artistry skills, which demands a professional knowledgeable in this domain. This paper presents a sandbox prototype for scene customization that allows importing, positioning, scaling, and saving changes for mobile robot simulation. Our preliminary testing of the sandbox has focused on usability to understand how the setting up of the environment is perceived. Preliminary results indicate that the sandbox is usable with improvements pertaining to improving the manipulation of the objects. © 2022 IEEE.

A Comprehensive Study to Analyze Trends in Web Search Interests Related to Fall Detection Before and After COVID-19

Falls, considered a serious health-related concern for the elderly people, are associated with multiple diverse and dynamic needs for the elderly people themselves, their caregivers, their family members, and healthcare professionals. The modern-day Internet of Everything lifestyle is characterized by people using the internet for a multitude of reasons which also includes seeking and sharing information related to such needs. Such activity on the internet results in the generation of tremendous amounts of web behavior-based Big Data which can be studied and analyzed to investigate the trends in the underlining needs and the associated web search interests. The COVID-19 pandemic that the world is facing right now has impacted the elderly population to a significant extent. In fact, the elderly population is considered a demographic group that is most likely to get infected by this virus and develop serious symptoms, which could lead to hospitalizations and death. There hasn't been any study conducted in the field of aging research thus far that investigates how the COVID-19 pandemic may or may not have impacted the needs related to fall detection in the elderly. This work aims to address this research challenge. A dedicated methodology based on Google Trends is proposed in this paper that studies the web behavior-based Big Data related to fall detection from different countries both before and after the pandemic. The preliminary results presented from the analysis of the web behavior-based Big Data from 14 countries - USA, India, Germany, United Kingdom, Spain, Australia, Indonesia, Malaysia, Thailand, South Africa, Canada, Philippines, Sweden, and Ireland, which are amongst the countries worst hit by COVID-19, shows evidence that the pandemic had an impact towards increasing the web search interests related to fall detection in multiple countries. © 2022 ACM.

An Analysis of Body Language of Patients Using Artificial Intelligence.

In recent decades, epidemic and pandemic illnesses have grown prevalent and are a regular source of concern throughout the world. The extent to which the globe has been affected by the COVID-19 epidemic is well documented. Smart technology is now widely used in medical applications, with the automated detection of status and feelings becoming a significant study area. As a result, a variety of studies have begun to focus on the automated detection of symptoms in individuals infected with a pandemic or epidemic disease by studying their body language. The recognition and interpretation of arm and leg motions, facial recognition, and body postures is still a developing field, and there is a dearth of comprehensive studies that might aid in illness diagnosis utilizing artificial intelligence techniques and technologies. This literature review is a meta review of past papers that utilized AI for body language classification through full-body tracking or facial expressions detection for various tasks such as fall detection and COVID-19 detection, it looks at different methods proposed by each paper, their significance and their results.

Home-based Self-health Management Strategies of COVID-19 for the Elderly in Applied Economics

At this moment of the COVID-19 epidemic, it is difficult for caregivers to be fully aware of the elderly by closing care to prevent accidents at home. Existing research, home-based self-health management strategies, by using contextual tools and a lack of empirical procedures or technological components in internet monitoring, home accidents from individualized patterns has not been achieved. We use vision detecting through the internet monitoring method in a smart lighting materials house to fill this research gap. We examined the impact of physical transitions and visibility on fall detection and compared the accuracies of fall prediction based on combinations of related factors. The results indicated that including both physical transitions and visibility would enable older people to avoid falls. We evaluated the impact of physical transitions and visibility on fall detection and compared the accuracy of falls based on combinations of related factors. The accuracy of predictions using both physical transition and visibility was higher than 81%, which is a high forecasting accuracy rate. Those are significant contributions to the elderly in applied economics.

An Efficient Patient Activity Recognition using LSTM Network and High-Fidelity Body Pose Tracking

The need for healthcare services is growing, particularly in light of the COVID-19 epidemic's convoluted trajectory. This causes overcrowding in medical facilities, making it difficult to manage, treat, and monitor patients' health. Therefore, a method to remotely observe the patient's behavior is required, to aid in early warning and treatment, and to reduce the need for hospitalization for patients with minor diseases. This paper proposes a new real-time smart camera system to monitor, recognize and warn the patient's abnormal actions remotely with reasonable cost and easy to deploy in practice. The key benefit of the proposed methods is that patient actions may be detected without the usage of ambient sensors by employing pictures from a regular video camera. It carries out the detection using high-fidelity human body pose tracking with MediaPipe Pose. Then, the Raspberry Pi 4 device and the LSTM network are used for remote monitoring and real-time classification of patient actions. The test dataset is built from reality and reuses the existing datasets. Our system has been evaluated and tested in practice with over 96.84% accuracy, runs at over 30 frames per second, suitable for real-time execution on mobile devices with limited hardware configuration.

Tiredness Detection for a Robotic Mobility Aid

The proportion of elderly people in society is predicted to continue to rise in the coming decades. Mobility is a key aspect of many daily activities, but falls become an increasingly significant health risk with age. With the COVID-19 pandemic, many elderly users prefer or require assistive devices, rather than human support, in walking and carrying out daily tasks. However, prior work has shown that when using passive assistive mobility devices, fall risks can actually increase. This presents an opportunity for assistive robots to help maintain and improve the mobility of elderly users, with an additional emphasis on safety, made possible through sensing capabilities. In this paper, we present a computer vision system that detects the eye blink and face angle patterns for exhibiting signs of tiredness. In addition to the frame-based detection, we also introduce a time-window collation with a machine learning classifier. The system proposed here is critical in monitoring the user, performing real-time detection, and recommending they take a break if tiredness is detected. The overall system architecture and algorithmic details are presented, then a series of experiments are conducted to validate the performance of the approach. © 2022 IEEE.

AnkFall-Falls, Falling Risks and Daily-Life Activities Dataset with an Ankle-Placed Accelerometer and Training Using Recurrent Neural Networks.

Falls are one of the leading causes of permanent injury and/or disability among the elderly. When these people live alone, it is convenient that a caregiver or family member visits them periodically. However, these visits do not prevent falls when the elderly person is alone. Furthermore, in exceptional circumstances, such as a pandemic, we must avoid unnecessary mobility. This is why remote monitoring systems are currently on the rise, and several commercial solutions can be found. However, current solutions use devices attached to the waist or wrist, causing discomfort in the people who wear them. The users also tend to forget to wear the devices carried in these positions. Therefore, in order to prevent these problems, the main objective of this work is designing and recollecting a new dataset about falls, falling risks and activities of daily living using an ankle-placed device obtaining a good balance between the different activity types. This dataset will be a useful tool for researchers who want to integrate the fall detector in the footwear. Thus, in this work we design the fall-detection device, study the suitable activities to be collected, collect the dataset from 21 users performing the studied activities and evaluate the quality of the collected dataset. As an additional and secondary study, we implement a simple Deep Learning classifier based on this data to prove the system's feasibility.

An Overview of Signal Processing Techniques for Remote Health Monitoring Using Impulse Radio UWB Transceiver.

Non-invasive remote health monitoring plays a vital role in epidemiological situations such as SARS outbreak (2003), MERS (2015) and the recently ongoing outbreak of COVID-19 because it is extremely risky to get close to the patient due to the spread of contagious infections. Non-invasive monitoring is also extremely necessary in situations where it is difficult to use complicated wired connections, such as ECG monitoring for infants, burn victims or during rescue missions when people are buried during building collapses/earthquakes. Due to the unique characteristics such as higher penetration capabilities, extremely precise ranging, low power requirement, low cost, simple hardware and robustness to multipath interferences, Impulse Radio Ultra Wideband (IR-UWB) technology is appropriate for non-invasive medical applications. IR-UWB sensors detect the macro as well as micro movement inside the human body due to its fine range resolution. The two vital signs, i.e., respiration rate and heart rate, can be measured by IR-UWB radar by measuring the change in the magnitude of signal due to displacement caused by human lungs, heart during respiration and heart beating. This paper reviews recent advances in IR- UWB radar sensor design for healthcare, such as vital signs measurements of a stationary human, vitals of a non-stationary human, vital signs of people in a vehicle, through the wall vitals measurement, neonate's health monitoring, fall detection, sleep monitoring and medical imaging. Although we have covered many topics related to health monitoring using IR-UWB, this paper is mainly focused on signal processing techniques for measurement of vital signs, i.e., respiration and heart rate monitoring.